PReMo : An Analyzer for P robabilistic Re cursive Mo dels

This paper describes PReMo, a tool for analyzing Recursive Markov Chains, and their controlled/game extensions: (1-exit) Recursive Markov Decision Processes and Recursive Simple Stochastic Games. Introduction. Recursive Markov Chains (RMCs) [4, 5] are a natural abstract model of probabilistic procedural programs and other systems involving recursion and probability. They are formally equivalent to probabilistic Pushdown Systems (pPDSs) ([2, 3]), and they define a class of infinite-state Markov chains that generalize a number of well studied stochastic models such as Stochastic Context-Free Grammars (SCFGs) and Multi-Type Branching Processes. In a series of recent papers ([4–7]), the second author and M. Yannakakis have developed algorithms for analysis and model checking of RMCs and their controlled and game extensions: 1-exit Recursive Markov Decision Processes (1-RMDPs) and 1-exit Recursive Simple Stochastic Games (1-RSSGs). These extensions allow modelling of nondeterministic and interactive behavior. In this paper we describe PReMo, a software tool for analysing models based on RMCs, 1-RMDPs, and 1-RSSGs. PReMo allows these models to be specified in several different input formats, including a simple imperative-style language for specifying RMCs and RSSGs, and an input format for SCFGs. For RMCs/RSSGs, PReMo generates a graphical depiction of the model, useful for visualizing small models (see Figure 1). PReMo has implementations of numerical algorithms for a number of analyses of RMCs and 1-RSSGs. From an RMC, PReMo generates a corresponding system of nonlinear polynomial equations, whose Least Fixed Point (LFP) solution gives precisely the termination probabilities for vertex-exit pairs in the RMC. For 1-RSSGs, it generates a system of nonlinear min-max equations, whose LFP gives the values of the termination game starting at each vertex. Computation of termination probabilities is a key ingredient for model checking and other analyses for RMCs and pPDSs ([4, 5, 2]). PReMo provides a number of optimized numerical algorithms for computing termination probabilities. Methods provided include both dense and sparse versions of a decomposed Newton’s method developed in [4], as well as versions of value iteration, optimized using nonlinear generalizations of Gauss-Seidel and SOR techniques. The latter methods also apply to analysis of 1-RSSGs. In addition to computing termination probabilities, PReMo can compute the (maximum/minimum/game) expected termination time in 1-RMCs, 1-RMDPs, and 1-RSSGs. It does so by generating a different monotone system of linear